# GQA attention mechanism

Hunyuan-7B is an open-source large language model released by Tencent. It has the ability to process 256K long texts and uses the Grouped Query Attention (GQA) mechanism, performing excellently among Chinese 7B dense models.

Hunyuan-7B-Instruct is a bilingual (Chinese-English) large language model released by Tencent, featuring powerful text generation and comprehension capabilities, and is one of the strongest Chinese 7B Dense models currently available.

Hunyuan 7B is an open-source bilingual large language model (Chinese and English) by Tencent, featuring optimized data ratios and training methods for robust performance, making it one of the strongest Chinese 7B Dense models available.

Mistral-NeMo-Minitron-8B-Base is a basic text generation model obtained by pruning and distilling Mistral-NeMo 12B, suitable for various natural language generation tasks.